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Abstract 

Previous  studies  have  established  a  link  between  Human  Capital  and  perfonnance 
both  at  the  firm  and  the  individual  level.  These  studies  have  shown  that  performance  can 
be  improved  with  additional  personnel  and/or  higher  education  levels.  This  study 
attempts  to  build  on  this  relationship  by  using  the  Cobb-Douglas  Production  function  to 
relate  inputs  to  outputs.  The  inputs  to  the  function  are  the  number  of  cost  analysts 
positions,  military  and  civilian,  and  an  education  variable  for  the  number  of  master’s 
degrees  in  the  field.  The  measure  of  output  is  the  average  cost  overrun  of  Air  Force 
contracts.  A  time  series  regression  was  conducted  while  controlling  for  other  economic 
factors  such  as  budgetary  fluctuations  and  inflation.  The  results  show  positive  effects  of 
human  capital  on  performance.  Other  policy  implications  of  the  study  are  the  importance 
of  budgetary  stability,  inflation  predictions  and  the  Defense  Acquisition  Workforce 
Improvement  Act  (DAWIA). 
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THE  IMPACT  OF  HUMAN  CAPITAL  ON  THE  COST  OF  AIR  FORCE 
ACQUISITION  PROGRAMS 

I.  INTRODUCTION 

The  primary  purpose  of  this  research  was  to  determine  if  there  is  a  relationship 
between  the  number  of  cost  analysts,  their  education  and  training  levels  and  the  cost 
variance  of  Air  Force  contracts.  In  this  study,  the  cost  variance  measurement  was  the 
amount  of  cost  overruns.  The  research  looked  at  both  officers  and  civilians  within  the 
cost  analysis  career  field. 

General  Issue 

One  of  the  ongoing  struggles  within  the  Department  of  Defense  (DoD) 
Acquisition  community  is  how  to  produce  weapon  systems  within  specified  budgets. 
There  have  been  several  studies  conducted  to  determine  the  actual  amount  of  cost 
variance  that  has  occurred  within  DoD  acquisition  systems.  Depending  on  the  time 
period  analyzed  and  the  type  of  variance  used  in  detennining  a  cost  difference,  these 
studies  found  that  costs  have  increased  by  an  average  of  8%  to  40%  (Jacques  S.  Gansler 
1989;  Jeffrey  A.  Drezner,  Jeanne  M.  Jarvaise,  Ron  Hess,  Daniel  M.  Norton,  and  Paul  G. 
Hough  1993;  David  S.  Christensen  and  James  A.  Gordon  1998;  William  J.  Swank,  Paul 
A.  Alfieri,  Charles  K.  Gailey  III,  and  Raymond  W.  Reig  2000;  James  Smirnoff  2006). 
Typically,  the  increased  costs  results  in  a  request  for  additional  funds,  but  in  extreme 
cases,  inaccurate  estimates  can  lead  to  program  cancellations,  and  the  loss  of  billions  of 
dollars,  as  was  the  case  for  the  A- 12,  a  controversial  program  that  was  cancelled  due  to 
overly  optimistic  estimates  (Christensen  1994). 
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Because  of  the  amount  of  money  involved  with  DoD  projects,  multi -billions  of 
dollars,  this  issue  has  received  significant  attention  from  senior  leaders  in  the  DoD,  and 
the  legislative  and  executive  branches  of  government.  As  such,  various  reform  measures 
have  been  instituted  over  the  past  40  years.  The  empirical  studies  on  the  effectiveness  of 
these  reforms  have  found  mixed  results.  Drezner  (1993)  looked  at  197  acquisition 
programs  from  1960  to  1990  and  found  no  improvement  in  cost  variance  due  to  the 
reforms.  In  addition,  David  S.  Christensen,  David  A.  Searle  and  Caisse  Vickery  (1999) 
found  that  one  of  the  biggest  refonn  measures,  the  Packard  Commission,  had  no  impact 
on  cost  variance.  Smirnoff  (2006),  however,  found  that  when  controlling  for  such  macro- 
economic  factors  as  annual  procurement  budget,  research  and  development  budget,  and 
unexpected  inflation,  some  of  the  reforms  have  been  effective  at  controlling  costs. 

There  have  been  several  studies  to  date  which  have  attempted  to  explain  where 
cost  variance  primarily  occurs  and  why  cost  estimates  are  inaccurate,  with  the  goal  of 
instituting  meaningful  reforms.  These  studies  have  determined  that  cost  variance  is 
primarily  influenced  by  the  characteristics  of  the  program  (program  size,  stage  of  the 
program,  urgency,  degree  of  testing,  and  production  rate)  and  the  characteristics  of  the 
weapon  system  (technological  maturity,  difficulty  of  technology,  type  of  commodity,  and 
performance  characteristics)  (Vince  Sipple,  Edward  White  and  Michael  Greiner  2004). 
These  studies  do  not  provide  a  policy  lever  that  can  be  used  to  reduce  cost  variance. 

They  only  identify  the  characteristics  of  programs  that  experience  an  increase  in  cost;  as  a 
result,  it  is  difficult  for  policy  makers  to  use  these  findings  to  prevent  the  unexpected 
increase  in  the  cost  of  weapon  systems.  More  needs  to  be  done  on  these  earlier  reports  to 
make  them  useful  for  decision  makers. 
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One  such  factor  which  could  influence  cost  variance  and  could  provide  a  tool  for 
policy  usage  is  the  quality  of  the  estimates.  The  idea  that  cost  overruns  are  attributed  to 
an  error  in  the  initial  cost  estimates  was  suggested  by  Christensen,  Searle  and  Vickery 
(1999)  and  Jeffrey  Drezner,  Jean  Jarvaise,  Ron  Hess,  Daniel  Norton,  and  Paul  Hough 
(1993).  Christensen,  Searle  and  Vickery  (1999)  noted  that  cost  realism  is  a  problem  in 
cost  estimation.  In  addition,  Bobby  J.  Pannel  (1994)  indicated  that  most  of  the  estimation 
errors  are  in  the  estimation  of  production  costs.  Pannel  identified  estimation  errors  as 
those  which  are  not  attributable  to  any  program  decisions  made  in  the  production  stage. 

While  these  studies  simply  identify  the  problem,  some  private  sector  studies  go 
deeper  into  the  issue.  Even  though  these  studies  are  not  specific  to  DoD  cost  estimating,  a 
general  cost  estimating  analogy  can  be  drawn.  The  two  types  of  studies  are  those  relating 
the  number  of  personnel  to  performance  (Xi  Li  2002)  and  those  relating  human  capital  to 
performance  (Gary  Becker  1964;  Anne  Bartel  1994;  Lisa  Lynch  and  Sandra  Black  1995; 
Alan  Krueger  and  Cecilia  Rouse  1998).  These  studies  concluded  that  the  performance  of 
the  firm  (Bartel  1994  used  net  sales)  and  of  the  individual  (Krueger  and  Rouse  1998  used 
performance  awards,  job  upgrades,  job  bids  and  job  attendance)  can  be  improved  with 
more  personnel  or  higher  education  levels.  In  addition,  two  studies  (Lynch  and  Black 
1995;  J.H.  Bishop  1994)  found  that  education  produces  indirect  benefits  such  as 
innovativeness  and  the  ability  to  better  adapt  to  new  tasks  and  technologies. 

Smirnoff  (2006)  evaluated  the  effect  of  the  Defense  Acquisition  Workforce 
Improvement  Act  of  1990  (DAWIA)  which  mandated  certain  education  and  training 
requirements  for  acquisition  personnel.  In  his  model,  he  found  that  this  act  actually  had  a 
positive  correlation  with  cost  overruns,  meaning  the  establishment  of  this  act  correlated 
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with  more  cost  overruns.  This  seems  counterintuitive;  so  this  variable  was  also  added  to 
this  study  to  see  if  different  results  are  produced  with  a  different  analysis  technique. 

With  the  exception  of  Smirnoff  (2006),  these  studies  suggest  estimation  error,  or 
the  perfonnance  of  Air  Force  cost  estimators,  could  be  improved  by  either  increasing  the 
amount  of  personnel  or  by  providing  more  education  to  those  already  employed.  If 
estimation  errors  are  reduced,  this  may  lead  to  fewer  cost  overruns  for  the  Air  Force. 
However,  the  current  state  of  the  Air  Force  makes  this  difficult.  According  to  the  Air 
Force  Cost  Analysis  Agency,  Air  Force  cost  analysis  resources  are  down  60%  (300 
positions)  since  1994,  while  the  total  AF  positions  are  down  13%  (Thomas  Dupre  2005). 
In  addition,  the  Air  Force  cost  field  has  experienced  cuts,  even  while  other  services  are 
increasing  their  cost  analysis  personnel;  Naval  Air  Systems  Command  (NAVAIR)  cost 
personnel  have  increased  91%  since  1992  and  NASA  cost  personnel  have  increased 
250%  (Dupre  2005).  Concurrently,  among  the  DoD,  the  Air  Force  is  historically  the 
poorest  performer  when  it  comes  to  cost  growth.  During  the  development  stage,  the  Air 
Force  experienced  a  78%  increase  in  cost  overruns,  compared  with  58%  for  the  DoD 
(Dupre  2005),  and  its  not  getting  any  better.  The  Air  Force  is  expecting  a  76%  growth  in 
costs  over  the  life  of  programs  currently  in  procurement  (Dupre  2005).  The  current 
feeling  in  the  Air  Force  Cost  Analysis  Agency  is  that  the  reason  the  Air  Force  is  doing 
poorly  compared  with  the  other  services  is  because  of  a  lack  of  cost  personnel  (Dupre 
2005). 

The  FY06  National  Defense  Authorization  Act  -  House  Armed  Services 
Committee  Report  has  directed  the  Air  Force  to  correct  the  situation  and  for  the  Secretary 
of  the  Air  Force  to  take  steps  necessary  to  correct  the  problems  in  cost  analysis  (Dupre 
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2005).  The  Secretary  of  the  Air  Force  reasoned  that  an  immediate  fix  to  the  cost  overrun 
problem  is  to  increase  the  number  of  cost  analysts  in  the  Air  Force.  The  Air  Force  has 
been  approved  to  add  50  more  personnel  to  the  career  field.  This  study  will  attempt  to 
determine  if  adding  more  personnel  will  indeed  fix  the  problem  or  if  the  cost  analysis 
career  field  is  already  at  a  suitable  strength  level. 

The  rest  of  this  study  will  be  organized  as  follows;  chapter  2  provides  a  review  of 
the  literature  related  to  this  field  of  study,  chapter  3  uses  this  literature  and  develops  a 
method  for  analyzing  the  relationship  between  cost  overruns  and  human  capital,  chapter  4 
provides  the  results  of  the  analysis  with  a  discussion  of  the  significant  findings,  and 
finally,  chapter  5  provides  additional  discussion  on  the  topic  and  provides  areas  for  future 
research. 
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II.  LITERATURE  REVIEW 


The  purpose  of  this  research  was  to  determine  whether  or  not  a  relationship  exists 
between  the  number  of  AF  cost  analysts,  both  military  and  civilian,  investments  in  human 
capital,  and  AF  contract  cost  variances.  In  order  to  give  a  background  of  the  topic,  this 
chapter  provides  an  overview  of  the  previous  research  that  has  been  conducted  in  this 
area. 

Cost  Variance 

There  are  two  methods  that  measure  cost  variance,  which  is  a  cost  increase  or 
decrease  in  a  program,  namely,  cost  growth  and  cost  overruns.  We’ll  reserve  discussion 
of  the  specifics  of  each  for  the  end  of  the  chapter;  however,  Vince  Sipple,  Edward  White, 
and  Michael  Greiner  (2004)  published  a  study  which  synthesized  all  research  on  cost 
variance  to  date.  In  the  article,  the  authors  summarized  studies  from  the  Balistic  Missile 
Defense  Organization  Study  in  2000,  a  1993  RAND  study,  a  NAVAIR  study  in  2001,  a 
study  by  Christensen  and  Templin  in  2000,  a  study  by  Henry  Eskew  in  2000,  an  Institute 
for  Defense  Analysis  (IDA)  study  in  1994,  and  a  2001  RAND  study.  In  their  review, 
Sipple,  White  and  Greiner  (2004)  noted  that  while  these  studies  only  identify  where  cost 
variance  occurs,  they  are  the  starting  point  to  identify  a  model  that  allows  cost  variance  to 
be  predicted  and  controlled. 

Estimation  error  is  one  factor  that  has  been  highlighted,  almost  as  an  afterthought, 
in  several  studiesv  The  Office  of  the  Secretary  of  Defense  Cost  Analysis  Improvement 
Group  (OSD  CAIG  2001)  did  an  extensive  examination  of  the  causes  of  cost  variances. 

In  this  examination,  the  underlying  reasons  seemed  to  result  from  either  decisions  or 
mistakes  that  were  made  regarding  the  program.  Decisions  that  drove  variances  included 
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requirement  changes,  schedule  changes,  and  changes  in  acquisition  strategy.  Estimation 
mistakes  included  assumption  or  estimation  changes,  schedule  slips  not  attributable  to 
technical  problems  and  any  other  changes  not  made  as  a  result  of  a  discretionary 
decision.  In  sum,  the  OSD  attributed  more  than  half  of  cost  variance  to  errors. 
Christensen,  Searle  and  Vickery  (1999)  focused  on  the  idea  of  errors  or  mistakes  when 
they  concluded  the  costs  are  not  only  underestimated  initially,  but  all  subsequent 
estimates  of  the  final  program  cost  are  underestimated.  The  authors  attribute  this  to 
estimation  error  or  other  causal  factors  not  identified.  Pannel  (1994)  suggested  some  of 
those  additional  factors,  indicating  that  the  majority  of  estimation  errors  are  due  to  errors 
in  the  production  cost  estimations.  These  errors  in  the  estimation  of  production  costs 
arise  because  of  method  errors,  omissions,  schedule  slips  attributable  to  technical 
problems,  weight  growth,  and  inadequately  scoped  engineering  and  software 
development  efforts  (Pannel  1994).  Sipple,  White,  and  Greiner  (2004)  diverged  from 
this  type  of  thinking  by  suggesting  that  costs  might  be  controlled  with  additional 
manpower  and  training. 

Human  Capital 

The  idea  that  human  capital  can  affect  perfonnance  is  something  that  has  been 
explored  in  the  private  sector.  Richard  Blundell,  Lorraine  Dearden,  Costas  Meghir,  and 
Barbara  Sianesi  (1999)  state  there  are  three  components  to  human  capital:  “early  ability 
(whether  acquired  or  innate);  qualifications  and  knowledge  acquired  through  formal 
education;  and  skills,  competencies  and  expertise  acquired  through  training  on  the  job” 
(p.  2).  They  go  on  to  assert  that  investing  in  human  capital  is  the  same  as  investing  in 
other  capital  assets.  An  up-front  fee  is  incurred  for  the  tuition,  loss  of  wages,  or  training 
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fees  but  a  return  on  investment  is  expected  (Blundell,  Dearden,  Meghir,  Sianesi  1999). 
Specifically,  this  study  found  that  education  and  training  have  an  important  impact  on  the 
individual,  firm  and  economy.  Past  findings  do  support  the  sequence  of  relationships  that 
are  hypothesized.  That  is,  studies  have  related  the  number  of  personnel  to  performance 
(Li  2002;  James  Shaw  and  Jeff  Weekley  1985).  Others  have  conducted  studies  to 
determine  how  much  return  can  be  expected  from  investing  in  training  or  education 
(Bishop  1994;  Krueger  and  Rouse  1998;  Bartel  1989,  1994;  Lynch  and  Black  1995) 

Numbers  of  Personnel 

Several  studies  have  concluded  that  the  number  of  personnel  making  the 
estimation  can  have  a  positive  effect  on  the  quality  of  the  estimation.  In  analyzing  the 
performance  of  stock  analysts,  Li  (2002),  for  instance,  found  the  analysts’  predictive 
power  began  to  decline  as  their  coverage  exceeded  12-13  stocks.  At  some  point  if  a 
worker  takes  on  too  much  work,  his  or  her  performance  begins  to  decline.  The  analyst 
does  not  have  time  to  conduct  a  thorough  review  of  the  stocks  and  their  estimates  become 
inaccurate.  The  same  could  be  true  with  a  military  cost  analyst.  If  a  military  cost  analyst 
is  working  on  a  large  program  that  requires  several  estimates,  he  might  not  be  able  to 
conduct  as  comprehensive  review  as  possible  and  could  produce  results  which  are  less 
accurate.  In  addition,  there  are  other  factors  involved  with  being  overworked.  Shaw  and 
Weekley  (1985)  conducted  an  experimental  study  in  which  they  gave  subjects  word 
puzzles  to  solve  in  order  to  determine  the  effects  of  overload  and  underload  situations.  In 
their  study,  they  measured  perceived  pressure,  self-esteem,  task  enjoyment,  resentment, 
anxiety,  depression,  hostility,  and  performance  and  found  overload  situations  caused  the 
subjects  to  feel  more  pressure,  have  less  task  enjoyment  and  feel  more  depressed  (Shaw 


8 


and  Weekly  1985).  If  this  sort  of  situation  were  to  be  prolonged,  you  would  expect  it  to 
have  a  negative  effect  on  performance. 

Education  Levels  of  Personnel 

Performance  cannot  be  improved  by  simply  adding  more  personnel.  It  is  also 
important  that  the  right  people  are  doing  the  job  with  the  right  skill  set  and  education 
levels.  There  have  been  numerous  private  sector  studies  on  human  capital  and  the 
performance  of  firms.  Becker  (1964),  in  his  pioneering  work  on  human  capital,  stated  the 
most  important  investments  in  human  capital  are  education  and  training.  In  the  years 
since,  there  have  been  two  types  of  studies  which  relate  education  and  training  to 
performance:  those  at  the  individual  level  and  those  at  the  firm  level. 

Individual  Level  of  Analysis 

At  the  individual  level,  Bishop  (1994)  used  data  from  the  National  Federation  of 
Independent  Business  survey  to  look  at  the  impact  of  training  on  newly  hired  employees. 
The  results  of  the  study  found  training  raises  productivity  by  16%,  where  productivity 
was  measured  subjectively  with  a  survey  that  asked  employers  to  rate  the  productivity  of 
the  new  employees.  However,  as  noted  by  Lynch  and  Black  (1995),  this  method  is  a 
subjective  measure  of  perfonnance  and  is  not  comparable  across  firms  or  within  firms. 

Krueger  and  Rouse  (1998)  measured  perfonnance  objectively  by  using  data  from 
records  from  both  a  service  and  manufacturing  company,  that  were  supplemented  with 
subjective  measures.  They  found  training  was  positively  conelated  with  the  occurrence 
of  job  bids,  upgrades,  perfonnance  awards,  and  job  attendance  across  industries.  This 
could  make  the  argument  that  higher  education  levels  could  lead  to  better  performance,  or 
in  this  case  better  cost  estimations  and  lower  cost  overruns. 
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Firm  Level  of  Analysis 

Other  studies  have  looked  at  the  effect  of  training  and  education  on  the 
performance  at  the  firm  or  organizational  level.  Bartel  (1994)  uses  the  Cobb-Douglas 
production  function  to  estimate  the  impact  of  employee  training.  Her  effort  was  an  early 
attempt  to  address  the  gap  in  the  literature  on  the  effects  of  training  and  productivity  at 
the  firm  or  organizational  level,  because  most  of  the  prior  research  on  this  topic  was  done 
at  the  individual  level.  She  found  that  organizations  that  were  operating  below  the 
expected  productivity  increased  their  productivity  after  establishing  training  programs. 
These  findings  were  echoed  by  Lynch  and  Black  (1995). 

Lynch  and  Black  (1995),  again  applying  the  Cobb-Douglas  Production  function, 
used  data  from  the  National  Center  on  the  Educational  Quality  of  the  Workforce  and 
confirmed  the  influence  of  capital  investments  on  organizational  level  outcomes. 
Specifically,  they  found  that  a  10%  increase  in  the  average  education  levels  of  employees 
resulted  in  an  8.5%  increase  in  productivity  in  manufacturing  and  12.7%  in  non¬ 
manufacturing  firms. 

Bartel  (1989)  used  a  survey  on  human  resource  management  and  measures  of 
financial  performance  to  determine  a  correlation  between  training  and  productivity.  She 
concluded  training  increases  productivity  by  16%;  however,  the  low  6%  survey  response 
rate  could  place  the  actual  productivity  rate  at  a  different  number.  Also,  another  problem 
with  Bartel’s  1989  study  is  that  most  firms  used  in  her  study  were  multiple  establishment 
firms.  The  measure  of  output,  financial  performance,  was  for  the  firm  as  a  whole  and  not 
the  specific  locations.  Based  on  this  study,  there  could  be  a  problem  with  correlating 
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training,  which  is  an  input  at  the  specific  locations,  to  output,  which  is  a  measure  of  the 
entire  firm’s  performance  (Lynch  and  Black  1995). 

Indirect  Benefits  of  Education 

In  addition,  the  studies  of  education  show  there  are  indirect  benefits  to  education. 
Workers  with  higher  education  levels  can  learn  how  to  leam  (Lynch  and  Black  1995), 
can  adapt  to  new  tasks  and  technologies,  and  are  more  innovative  (Bishop  1994). 
Specifically,  Bishop  (1994)  found  that  the  number  of  years  schooled  increased  an 
innovation  measure  by  7.8%. 

Summary 

This  chapter  showed  there  is  a  historical  link  between  human  capital  and 
performance  of  both  the  individual  and  the  firm.  The  next  chapter  will  use  this  link  to 
build  a  model  to  test  the  relationship  between  Air  Force  cost  analysis  human  capital  and 
cost  overruns. 
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III.  METHOD 


In  order  to  deal  with  some  of  the  shortcomings  in  the  literature  linking  human 
capital  to  performance,  this  study  will  use  an  organizational  level  of  analysis  and  an 
objective  measure  of  performance.  The  method  of  analysis  will  be  the  Cobb-Douglas 
production  function  similar  to  Bartel  (1994).  In  addition,  the  performance  measure  will 
be  the  amount  of  cost  variance  per  year.  Following  is  a  description  of  how  this  cost 
variance  will  be  captured. 

Measuring  Contract  Performance 

The  performance  of  contracts  is  determined  by  various  factors  such  as  its  schedule 
and  whether  or  not  the  final  product  met  the  requirements  of  the  customer.  One 
performance  factor  which  receives  a  lot  of  scrutiny  is  cost.  There  are  two  methods  that 
measure  a  cost  increase  or  decrease  in  a  program,  namely,  cost  growth  and  cost  overrun. 
Cost  growth  is  calculated  by  subtracting  the  initial  estimate  from  the  final  program  cost, 
while  a  cost  overrun  is  computed  by  subtracting  the  budgeted  cost  of  work  performed 
(including  budget  changes)  from  the  actual  cost  of  work  perfonned  (Christensen  and 
Gordon  1998).  This  is  also  consistent  with  the  definition  provided  by  the  Earned  Value 
Management  System  (Earned  Value  Management  2006).  The  appropriate  measure  (i.e., 
cost  growth  or  overrun)  varies  based  on  the  researcher’s  purpose.  When  attempting  to 
identify  errors  in  estimation,  a  cost  overrun  is  the  better  measurement.  Factors  which  are 
out  of  the  program’s  control,  such  as  funding  instabilities  and  changes  in  the  scope  of  a 
requirement,  can  lead  to  cost  growth  but  not  to  cost  overruns  (Christensen  and  Gordon 
1998).  The  final  budget,  which  is  used  in  the  cost  overrun  calculation,  is  a  better 
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measure  of  how  well  a  program’s  budget  was  estimated.  Figure  1  captures  the  overall 
cost  variance  computation  (Earned  Value  Management  2006) 


Reporting  Estimated  Cost 

Cut-off  Date  at  Completion 


Projected 

Overrun 


Mgt  Res 

Budget  at 
Completion 


Figure  1 :  NASA  Earned  Value  Management  (2006) 

The  actuals  line,  also  known  as  the  Actual  Cost  of  Work  Performed  (ACWP) 
reflects  the  amount  spent  to  date,  or  the  amount  paid  to  the  contractor  The  plan  line,  also 
called  the  Budgeted  Cost  of  Work  Scheduled  (BCWS),  is  the  planned  expenditures,  or 
the  budget  of  the  contract.  The  earned  value  line  is  the  Budgeted  Cost  of  Work 
Performed  (BCWP)  and  is  the  value  of  completed  work.  The  value  of  the  completed 
work  is  what  takes  into  account  requirement  and  budget  changes.  A  cost  variance  is 
computed  as  the  difference  between  the  ACWP  and  the  BCWP.  In  other  words,  it  is  the 
difference  between  what  has  been  spent  and  what  has  actually  been  accomplished. 
Because  this  measurement  takes  into  account  the  budgeted  amount  (program  changes  and 
changes  in  quantity)  the  cost  variance  is  equivalent  to  a  cost  overrun.  Cost  growth,  on  the 
other  hand,  would  be  the  difference  between  the  ACWP  line  and  the  initial  budget  (not 
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shown  on  the  graph).  This  form  of  measuring  cost  variance  would  not  take  into  account 
re-baselines  of  the  budget  or  quantity  changes,  and  so  it  is  not  a  good  measure  of 
estimation  accuracy. 

Data  and  Variables 

Dependent  Variable 

To  conduct  this  study,  data  was  collected  on  the  average  cost  overruns  of  Air 

Force  weapon  systems  per  year  from  1970  to  2006  using  Earned  Value  Management  data 

(EVMS)  from  the  Defense  Acquisition  Executive  Summary  (DAES).  The  amount  of 

overrun  for  each  weapon  system  will  be  consolidated  by  year  to  give  the  average  cost 

overrun  for  that  year  in  percentage  terms  (Earned  Value  Management  2006): 

%Overrun=  (ACWP-BCWP)  *  100  (1) 

(BCWP) 


Independent  Variables 

Personnel  numbers  and  education  levels  were  collected  from  the  Air  Force 
Personnel  Center,  the  agency  responsible  for  maintaining  personnel  records  for  each 
occupational  Air  Force  specialty.  The  numbers  represent  those  identified  as  cost 
analysts,  using  a  specific  occupational  code  within  the  Air  Force  personnel  system 
(officers  having  the  Air  Force  duty  occupational  code  65W).  Because  there  is  no  Cost 
Analysis  career  field  for  civilians,  an  estimate  has  to  be  made  as  to  the  actual  numbers 
that  are  performing  cost  analysis  duties.  The  civilians  in  cost  analysis  come  from  two 
career  fields,  Occupational  Series  501,  Financial  Management  and  Program,  and  1515, 
Operations  Research.  Not  all  civilians  in  these  occupational  series  perfonn  cost  duties, 
so  we  eliminated  all  but  those  civilians  funded  within  the  Program  Element  Code  (PEC) 
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72806K,  Acquisition.  It  can  be  assumed  that  civilians  in  this  PEC  who  are  in  the  501 
series  and  1515  series  are  doing  work  related  to  cost  analysis.  This  method  was 
crosschecked  with  the  current  civilian  cost  analysts  manning  levels  on  record  at  the  Air 
Force  Cost  Analysis  Agency  and  produced  similar  results. 

Similar  to  the  work  done  by  Smirnoff  (2006),  a  dummy  variable  for  training  was 
added.  Starting  in  1994,  the  dummy  variable  captures  the  effects  when  all  the  provisions 
of  the  DAWIA  Act  of  1990  were  put  in  place  (William  J.  Perry  1995).  Smirnoff  (2006) 
found  a  positive  correlation  of  this  act  on  cost  overruns.  This  research  will  determine  if 
this  same  finding  holds  true  using  a  Cobb-Douglas  function. 

Control  Variables 

In  order  to  account  for  other  factors  which  could  contribute  to  the  amount  of  cost 
overruns  per  year,  control  variables  were  added  to  the  model.  The  Air  Force  Research 
and  Development  budget  per  year  was  added  as  a  technological  proxy.  One  of  the 
common  criticisms  of  inaccurate  estimates  is  that  new  weapon  systems  are  becoming 
more  complex  and  thus  are  more  difficult  to  estimate.  If  this  is  the  case,  more  complex 
systems  with  higher  technologies  will  require  more  Research  and  Development  funds. 

Of  course,  if  the  number  of  R&D  programs  is  increased,  this  would  also  cause  the  R&D 
budget  to  increase.  So  there  could  be  some  error  in  using  this  variable  as  a  proxy  for 
technology  level. 

Another  factor  to  control  for  is  the  amount  of  analytical  work  which  is  outsourced 
to  private  contractors.  Trying  to  detennine  the  actual  amount  of  contractors  who  are 
currently  doing  cost  estimates  is  a  difficult  task,  let  alone  trying  to  find  out  numbers  from 
previous  years.  In  order  to  estimate  this  and  provide  a  proxy  for  the  effects  of  contractor 
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work,  a  raw  number  for  the  total  number  of  people  doing  defense  related  work  in  private 
industry  was  added  to  the  model.  This  provides  a  trend  for  the  amount  of  analytical  work 
that  was  outsourced  per  year.  Given  that  this  is  an  estimate,  this  introduces  the 
possibility  for  errors  in  the  estimation. 

Because  this  research  is  dealing  with  estimation  error,  unexpected  inflation  is 
another  control  variable  we  added  to  the  model.  Because  of  the  large  amounts  of  dollars 
being  expended,  minor  changes  in  the  actual  amount  of  inflation  can  have  large  effects  on 
overruns. 

The  last  two  control  variables  are  the  total  budget  and  the  procurement  budget  of 
the  Air  Force.  These  last  two  control  variables  should  account  for  any  budget 
fluctuations  that  may  constrain  programs  and  cause  them  to  be  delayed,  and  consequently 
become  over  budget. 

Model 

The  basic  Cobb-Douglas  Production  function  is  given  by:  (Charles  Cobb  and  Paul 
Douglas  1928). 

Q  =  AKaPLaY  (2) 

Where  Q  is  a  measure  of  output,  A  is  a  technological  parameter,  K  is  capital,  and  L  is 
labor.  The  model  suggests  that  as  more  capital  or  more  labor  is  added  to  an  organization, 
the  output  increases.  This  model  was  adapted  by  simply  adding  another  input  variable, 
education  (Lynch  and  Black  1995). 

Q  =  KaPLaYEa^  (3) 

In  this  study,  Q  is  the  amount  of  cost  overruns,  K  represents  the  capital  of  the  defense 
industries,  L  is  reported  labor,  and  E  is  education.  L  is  simply  the  number  of  personnel  in 
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the  cost  analysis  career  field,  E  is  the  education  level  of  the  personnel  in  this  field. 

Finally,  we  conducted  a  log-log  transformation  on  equation  (3)  to  linearize  the  functional 
form  for  ease  of  estimation. 

In  Q  =  pink  +  ylnL  +  XlnE  +  aX  +  s  (4) 

The  natural  log  was  not  conducted  on  the  budget,  DAWIA  and  unexpected 
inflation  variables  due  to  the  existence  of  observations  with  negative  or  zero  values.  As 
such,  a  semi-log  interpretation  will  be  made  on  these  variables.  The  additional  variable  X 
is  added  to  represent  the  vector  of  control  variables.  In  the  Cobb-Douglas  Function, 
output  can  be  many  things  but  in  this  study  it  will  be  assumed  that  output  is  equal  to  cost 
overruns.  Bartel  (1994)  used  net  sales  as  a  proxy  for  output.  A  cost  overrun  is  similar  to 
a  negative  “net  sale”  for  the  Air  Force,  it  takes  away  funds  from  other  projects.  It  is  also 
assumed  that  the  capital  of  the  defense  industries,  specifically,  the  Industrial  Production 
of  the  Defense  Industries,  will  be  a  good  proxy  for  K,  since  this  study  is  looking  at 
contracts.  These  contracts  are  undertaken  by  the  defense  industries  using  in  large  part, 
their  own  capital,  not  the  Air  Force’s  capital.  The  use  of  Industrial  Production  data  is  a 
standard  proxy  capital  in  the  literature. 

While  there  is  a  technological  parameter  in  the  equation,  in  some  studies  using  the 
Cobb-Douglas  production  function,  this  is  omitted.  When  a  log  transformation  is 
conducted  on  the  Cobb-Douglas  function,  the  technological  parameter,  A,  becomes  the 
intercept  in  a  linear  equation.  Since  this  study  is  only  looking  at  the  relationship  between 
personnel,  education,  and  cost  overruns,  which  is  the  slope  of  the  Cobb-Douglas  function, 
the  technological  parameter  will  be  given  a  value  of  1,  with  no  loss  of  generality. 
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Summary 


This  chapter  provided  a  review  of  the  model  to  be  used  in  the  analysis,  how  it  was 
derived,  the  variables  being  considered,  why  they  are  important  and  how  they  were 
collected.  In  the  next  chapter,  the  results  of  the  analysis  will  be  presented. 
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IV.  RESULTS 


To  begin,  the  key  parameters  of  the  analysis,  cost  overruns  and  military  and 
civilian  analysts  were  plotted  against  time,  to  get  a  visual  representation  of  the  data.  Due 
to  the  range  of  the  civilian  data,  this  study  ranged  from  1988  through  2005. 
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Figure  2:  Cost  Overruns  vs.  Military  and  Civilian  Analysts 

Below  are  the  summary  statistics  of  the  database.  The  mean  cost  overrun  is 
9.79%  with  a  range  of  0.284%  to  25.71%.  The  number  of  military  and  civilian  analyst 
range  from  487  to  834  with  a  mean  of  589.  The  percentage  of  MA/MS  degrees  averaged 
at  47%  and  ranged  from  41.5%  to  53.5%.  This  information  will  be  used  later  in  the 
chapter  in  the  interpretation  of  the  significant  coefficients. 
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Variable 

Obs 

Mean  Std. 

Dev.  Min 

Max 

Overrun  Percent 

17 

9.79 

10.17 

0.284 

27.716 

AF  Budget  Change  (Lag  2) 

17 

-0.01 

0.08 

-0.095 

0.228 

R&D  Budget  Change  (Lag  2) 

17 

0.003 

0.10 

-0.154 

0.287 

Procurement  Change  (Lag  2) 

17 

-0.001 

0.14 

-0.212 

0.322 

Percent  MA/MS 

17 

47.00 

3.32 

41.487 

53.541 

Capital 

17 

112.26 

16.29 

90.100 

148.500 

Number  of  Analysts 

17 

589.12 

122.87 

487 

834 

AFIT  Grads  (Lag  1) 

17 

11.29 

3.16 

6 

17 

Unexpected  Inflation 

17 

0.0002 

0.005 

-0.010 

0.010 

Defense  Employment 

17 

2777.94 

540.22 

2180 

3850 

DAWIA 

17 

0.71 

0.47 

0 

1 

Table  1:  Summary  Statistics 


Pre-Estimation  Specification  Tests 


In  order  to  conduct  a  time-series  regression  analysis,  both  the  dependent  and 


independent  variables  must  be  stationary.  Any  trends  or  drifts  in  the  variables  can  lead  to 


a  spurious  regression  and  faulty  conclusions.  If  two  non-stationary  variables  that  have  no 


relationship  (for  example  height  of  a  tree  and  GDP)  are  regressed  on  each  other,  it  will 


appear  as  if  the  height  of  the  tree  is  affecting  GDP.  The  common  procedure  for  ensuring 


stationarity  is  to  first  test  for  it  using  the  Augmented  Dickey-Fuller  test.  If  the  variable  is 


found  to  be  non  stationary,  the  variable  is  replaced  with  the  change  in  that  variable  from 


one  year  to  the  next,  called  the  first  difference.  Using  this  procedure,  the  dependent 


variable,  Number  of  Analysts,  Education,  Defense  Employment  and  Capital  variables 


required  a  first  difference  to  become  stationary. 


The  next  step  was  to  detennine  the  lag  lengths  of  the  independent  variables.  The 


theory  behind  lags  is  that  the  presence  of  an  independent  variable  may  not  be  felt  until 


later  time  periods  in  the  dependent  variable.  To  detennine  the  appropriate  lag  length, 


each  independent  variable  was  regressed  on  the  dependent  variable  with  the  results  of  the 


Akaike  Information  Criteria  (AIC)  and  adjusted  R  squared  shown  in  table  2.  Because 
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adding  an  additional  lag  decreases  the  amount  of  useable  observations  and  therefore 
causes  the  AIC  to  decrease,  the  AIC  was  divided  by  the  number  of  observations  to  give 
the  variance  per  observation;  this  was  crosschecked  with  the  R  value  of  the  regression. 
In  all  cases,  the  AIC  was  minimized  when  the  R  was  maximized,  except  for  the  capital 
variable.  As  such,  this  lag  was  defaulted  to  a  no  year  lag.  The  minimum  AIC  of  the 
procurement  variable  occurred  at  the  second  lag  with  an  R“  of  close  to  0.00.  Lag  0  and 
lag  3  of  this  variable  also  had  an  R“  of  close  to  zero.  As  such,  the  second  lag  was  chosen 
in  order  to  be  consistent  with  the  other  budget  variables.  All  variables,  with  the 
exception  of  AFIT  Graduates  which  had  a  lag  of  1,  and  the  budget  variables,  which  had  a 
lag  of  2,  were  maximized  at  no  lags.  The  2  year  lag  on  the  budget  changes  is  to  be 
expected.  When  budgets  change,  schedules  and  requirements  consequently  change  to 
free  up  additional  funds.  As  a  result,  an  immediate  effect  probably  would  not  be  felt.  It 
is  reasonable  to  assume  that  the  true  effects  of  a  budget  change  will  not  occur  until  later 
years. 


Cost  Analysts 

AFIT  Grads 

Capital 

DAWIA 

AF  Budget  Change 

AIC 

R2 

AIC 

R2 

AIC 

R2 

AIC 

R2 

AIC 

R2 

No  Lag 

0.08 

-0.04 

-0.02 

0.03 

0.00 

One  Lag 

2.80 

0.00 

2.48 

0.14 

2.30 

-0.03 

2.37 

0.03 

2.32 

Two  Lags 

2.91 

-0.06 

2.63 

0.05 

2.30 

-0.03 

2.36 

0.00 

2.23 

Three  Lags 

2.89 

-0.06 

2.78 

-0.05 

2.33 

-0.03 

2.37 

-0.03 

2.28 

Four  Lags 

2.93 

-0.02 

2.84 

-0.02 

2.37 

-0.04 

2.35 

-0.01 

2.27 

Five  Lags 

2.96 

0.03 

2.86 

-0.05 

2.40 

-0.04 

2.40 

-0.03 

2.28 

Education 

Defense  Employment 

Unexpected  Inlfation 

R&D  Budget  Change 

Proc.  Budget  Change 

AIC 

R2 

AIC 

R2 

AIC 

R2 

AIC 

R2 

AIC 

R2 

No  Lag 

a,*! 1  mi 

-0.02 

2.28 

2.33 

-0.01 

0.008 

One  Lag 

2.73 

0.07 

2.37 

-0.03 

2.33 

-0.03 

2.40 

-0.01 

2.33 

-0.03 

Two  Lags 

2.78 

0.07 

2.40 

-0.03 

2.35 

-0.03 

2.24 

0.02 

2.27 

-0.006 

Three  Lags 

2.90 

-0.06 

2.43 

-0.04 

2.37 

-0.01 

2.27 

0.00 

2.27 

-0.002 

Four  Lags 

2.93 

-0.02 

2.46 

-0.04 

2.41 

-0.05 

2.32 

-0.02 

2.32 

-0.03 

Five  Lags 

2.95 

0.04 

2.48 

-0.02 

2.41 

-0.01 

2.36 

-0.03 

2.34 

-0.01 

Table  2:  Lag  Structure  Determination 
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Post-Estimation  Specification  Tests 

With  the  configuration  of  the  dependent  and  independent  variables  determined,  the 
next  step  was  to  perform  diagnostic  checks  of  the  model.  Tests  for  multi-collinearity, 
heteroskedasticity,  auto-correlation,  cointigration,  nonnal  distribution  of  the  error  terms 
and  omitted  variable  bias  were  conducted. 

To  test  for  multi-collinearity,  the  Variance  Inflation  Factor  (VIF)  was  calculated,  see 
Appendix  A.  The  mean  VIF  for  the  variables  in  the  model  was  3.79.  Ideally,  this  value 
would  be  below  10  to  ensure  that  no  collinearity  was  present,  so  this  model  is  within  the 
safe  zone  for  not  having  multi-collinearity. 

In  order  to  test  for  heteroskedascity,  two  methods  were  used;  a  visual  inspection  as 
well  as  an  empirical  test  of  the  data.  To  empirically  test  for  heteroskedasticty,  the 
Breusch-Pagan/Cook-Weisberg  test  was  used.  With  a  p-value  of  0.4672,  it  is  not  possible 
to  reject  the  null  hypothesis  that  heteroskedasticity  is  present.  For  a  visual  inspection  of 
the  residuals,  see  Appendix  B. 

To  test  for  auto-correlation  of  the  residuals,  the  Durbin  Watson’s  d-statistics  and 
the  Durbin’s  alternative  test  for  autocorrelation  were  used.  The  d-statistic  was  2.36. 

With  17  observations  and  10  variables,  the  d-statistic  should  be  between  0.197  and  3.184 
in  order  to  not  be  auto-correlated.  As  such,  the  d-statistic  was  within  the  safe  zone.  The 
alternative  test  for  autocorrelation  yielded  a  prob>chi2  of  0.287,  with  the  null  being  no 
autocorrelation,  so  it  is  not  possible  to  reject  that  there  is  no  autocorrelation  present. 

The  next  test  was  to  ensure  there  was  no  cointigration  of  the  error  terms.  Even 
though  the  variables  are  stationary  or  made  stationary  by  first  differencing,  the 
combination  of  the  variables  could  be  creating  a  non-stationary  process.  The  Augmented 
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Dickey  Fuller  test  was  conducted  on  the  residuals  to  ensure  they  are  stationary.  The 
result  was  a  test  stastistic  of  -6.65,  with  a  1%  critical  value  of  -3.75,  giving  a  p-value  of 
0.000.  Therefore,  we  can  be  assured  that  the  residuals  are  stationary. 

The  last  test  is  to  ensure  that  the  error  terms  are  normally  distributed.  To  test  for 
this,  two  different  tests  were  conducted;  a  visual  inspection,  and  an  empirical  test.  The 
visual  inspection  of  the  residuals,  located  in  Appendix  C,  shows  that  the  error  terms 
appear  to  fairly  normally  distributed.  The  Shapiro-Wilk  test  revealed  that  the  degree  of 
skewness  and  kurtosis  were  significant  to  the  0.422  and  0.382  level  respectively,  with  a 
joint  prob>chi2  of  0.4549,  not  enough  to  conclude  that  the  residuals  are  not  normally 
distributed. 

The  Ramsey  RESET  test  was  used  to  test  for  omitted  variable  bias  and  for 
misspecification  of  the  model.  With  a  prob>chi2,  it  is  not  possible  to  reject  the  null 
hypothesis  that  the  model  has  no  omitted  variables.  In  addition,  exogeneity  was  not 
considered  an  issue  since  this  model  is  derived  from  the  theoretical  relationship  of  the 
Cobb-Douglas  Production  Function. 

Results 

The  results,  with  the  required  specification  changes  based  on  the  above  diagnostic 
tests,  are  displayed  in  table  3  below: 
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Number  of  obs  =  17 

R-squared  0.8365 

Adjusted  R-squared  0.5639 


1st  Diff  In  (Cost  Overrun  Percentage) 

Coef. 

Std.  Err. 

t 

P>|t| 

Proc.  Budget  Change  (Lag  2) 

2.86 

2.41 

1.19 

0.281 

R&D  Budget  Change  (Lag  2) 

3.23 

2.86 

1.13 

0.302 

Total  Budget  Change  (Lag  2)** 

-17.62 

5.41 

-3.25 

0.017 

1st  Diff  In  (Education)* 

-9.64 

4.53 

-2.13 

0.077 

Ln  (AFIT  Grads)  lag  1 

-0.50 

0.84 

CD 

© 

■ 

0.572 

1st  Diff  ln  (Capital) 

6.56 

3.78 

1.73 

0.134 

1st  Diff  ln  (Military/Civilian  Analysts)* 

-10.06 

4.86 

-2.07 

0.084 

DAWIA*** 

-2.21 

0.54 

-4.1 

0.006 

1st  Diff  ln  (Defense  Employment)** 

12.56 

4.46 

2.82 

0.030 

Unexpected  Inflation 

108.34 

62.31 

1.74 

0.133 

cons 

2.37 

1.95 

1.21 

0.270 

Notes  Table  3:  Time-Series  Regression  Results 

*Significant  to  the  10%  level 
**Significant  to  the  5%  level 
***Significant  to  the  1%  level 


According  to  the  results,  total  budget  change,  education,  number  of  analysts 
DAWIA  and  Defense  Employment  are  all  significantly  correlated  with  the  amount  of  cost 


overruns  in  a  given  year. 

The  results  show  that  adding  more  analysts  is  correlated  with  a  reduction  in  cost 
overruns.  Specifically,  for  every  1%  the  number  of  military  analysts  is  increased,  the 
cost  overrun  percentage  decreases  by  10%.  On  average,  there  are  597  cost  analysts  in  the 
AF  and  a  10%  contract  cost  overrun  occurs.  If  the  amount  of  military  analysts  is 
increased  by  1%,  or  6  analysts,  the  percentage  of  cost  overruns  could  be  expected  to 
decrease  by  (10*. 1)  =  1.0. 

Increasing  the  number  of  MA/MS  degrees  is  also  correlated  with  a  reduction  in 
cost  overruns.  For  every  1%  the  percentage  of  MA/MS  degrees  is  increased,  cost 
overruns  drop  by  0.0964%.  On  average,  there  are  46.7%,  or  279,  analysts  with  a  MA/MS 
degree.  If  this  number  is  increased  by  1%,  or  3  more  MA/MS  degrees,  cost  overruns 
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would  decrease  by  (10*. 00096)  =  0.0096.  Adding  30  more  MA/MS  degrees  could 
decrease  overruns  by  0.096. 

The  Air  Force  total  budget  was  also  a  significant  contributor  to  overruns. 
According  to  the  results,  for  every  1%  the  total  budget  is  increases,  the  amount  of  cost 
overruns  would  decrease  by  0.176%.  On  average,  with  a  10%  cost  overrun,  adding  1%  to 
the  budget  could  decrease  the  amount  of  overruns  by  (10*. 00 176)  =  0.0176.  This  makes 
a  strong  case  for  the  importance  of  budget  stability  in  acquisition  programs. 

The  proxy  for  defense  contractors,  defense  employment,  was  also  significant. 
However,  it  had  the  opposite  sign  that  would  be  expected.  The  coefficient  indicates  that 
if  the  amount  of  defense  employees  is  increased  by  1000,  overruns  would  actually 
increase  by  12.5%.  This  seems  counterintuitive.  A  possible  explanation  is  that  this 
variable  is  also  acting  as  a  size  variable.  As  the  Air  Force  gets  bigger,  we  incur  more 
overruns.  It  could  be  that  the  Air  Force  has  become  too  large  and  has  entered  into  an  area 
of  diseconomies  of  scale.  This  fact  could  also  be  seen  in  the  capital  variable.  Even 
though  this  variable  was  not  significant,  it  had  a  positive  sign  as  well  indicating  that  has 
the  Air  Force  gets  bigger,  more  overruns  follow. 

The  last  significant  variable  was  the  Defense  Acquisition  Workforce 
Improvement  Act.  This  variable  had  a  coefficient  of  -2.21.  Because  this  is  a  dummy 
variable,  transforming  this  into  a  meaningful  number  requires  the  use  of  the  equation 
(Patricia  A.  Champ,  Kevin  J.  Boyle  and  Thomas  C.  Brown  2003) 

g  =  (eAp-l)*100 

Where  g  is  the  percentage  effect  of  the  dependent  variable,  and  (3  is  the  coefficient  of  the 
variable.  The  resulting  percentage  effect  DAWIA  has  on  cost  overruns  is  8.9%.  This 
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would  mean,  on  average  with  a  cost  overrun  percentage  of  10%,  the  occurrence  of 
DAWIA  has  dropped  cost  overruns  by  0.89.  This  is  counter  to  other  studies  that  found  a 
positive  relationship  between  DAWIA  and  overruns.  It  could  be  that  when  controlling 
for  human  capital  factors,  DAWIA  has  actually  helped  to  reduce  the  amount  of  overruns. 

The  unexpected  inflation  variable  is  also  worth  mentioning.  This  variable  was 
marginally  significant  to  the  0.13  level.  However,  the  sign  of  the  variable  was  positive, 
as  you  would  expect,  indicating  that  as  the  amount  of  unexpected  inflation  increases,  cost 
overruns  increase.  The  magnitude  of  the  variable  indicated  that  if  unexpected  inflation 
increased  by  1%,  the  amount  of  overruns  increase  by  approximately  1%,  almost  a  1  to  1 
ratio. 

Some  of  the  magnitudes  of  the  coefficients  seem  relatively  small  but  when  you 
consider  the  amount  of  money  that  is  involved,  the  values  are  fairly  significant.  Table  4 
below  summarizes  the  significant  variables  and  puts  them  into  prospective  in  terms  of  the 
dollar  amounts  that  are  involved. 


Procurement  Budget  2006  $32. 6B 

R&D  Budget  2006  $21.1B 

Total  $53. 7B 

10%  Average  Overrun  0.1 

Total  Overruns  $5.37B 


Dollar  Amount 

$5,155,200.00 

$5,370,000.00 

$6,981,000.00 

$9,129,000.00 

$5.370,000.00 

$32,005,200.00 


Percentage  Effect 


Add  3  degrees  0.10% 

Add  6  Analysts  0. 1 0% 

Subtract  1000  Defense  Employees  0.13% 

Add  1%  to  Budget  0.17% 

Decrease  Unexpected  Inflation  1%  0.10% 

Total  percent  decrease  0.60% 


Table  4:  Associated  Dollar  Amounts  for  Significant  Coefficients 
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Given  that  the  2006  Air  Force  procurement  budget  was  $32. 6B  and  the  R&D 
budget  was  $21.  IB,  for  a  total  of  $53. 7B,  and  overruns  average  10%  per  year,  there  is  an 
overrun  of  $5.37B  per  year.  Multiplying  this  number  by  the  associated  percentage  effects 
of  the  significant  coefficients,  the  column  on  the  right  shows  how  much  money  is 
involved  with  these  changes.  It  can  be  seen  that  by  making  these  changes,  the  Air  Force 
could  either  save  or  more  accurately  budget  for  approximately  $32B  dollars.  It  is  yet  to 
be  detennined  if  this  is  an  actual  savings  or  just  money  that  would  be  more  accurately 
predicted.  More  discussion  of  this  will  follow  in  the  next  chapter. 
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V.  DISCUSSION 


An  area  of  future  research  could  be  to  optimize  the  number  of  cost  analysts  and 
their  education  levels  based  on  the  cost  of  acquiring  new  personnel,  sending  them  to 
school,  giving  them  training  and  finally  enough  experience  to  become  productive 
workers.  Taking  this  cost  into  account  and  comparing  it  to  the  expected  decrease  in  cost 
overruns,  a  better  decision  can  be  made  as  to  whether  or  not  the  Air  Force  will  ultimately 
do  better  with  more  analysts.  This  is  because  a  decrease  in  overruns  may  or  may  not 
coincide  with  an  actual  saving.  The  results  show  that  adding  more  analysts  can  decrease 
the  amount  of  overruns.  However,  this  could  just  indicate  that  we  are  better  able  to 
estimate  the  cost  of  the  weapon  system  and  therefore  would  not  incur  any  overruns.  The 
cost  of  the  weapon  system  remains  the  same;  we  just  better  predicted  the  price.  The 
greatest  benefit  would  occur  in  correctly  forecasting  budgets  and  allocating  resources.  It 
was  seen  that  budget  fluctuations  have  a  significant  effect  on  overruns.  If  we  can  more 
accurately  budget  resources,  we  can  subsequently  maintain  stable  budgets  for  acquisition 
programs  and  could  then  possibly  save  money. 

Also,  the  number  of  observations  in  the  study  was  relatively  low,  only  17.  One 
way  to  increase  the  number  of  observations  is  to  incorporate  the  other  services  into  the 
study.  This  would  greatly  enhance  the  credibility  of  the  results.  We  suspect  that  the 
same  results  will  hold  if  the  other  services  are  added  since  it  appears  the  same 
relationship  exists;  it  was  seen  that  the  other  services  are  building  up  their  cost  analysis 
manpower  and  at  the  same  time  they  are  experiencing  less  overruns  than  the  Air  Force. 

The  effect  of  budget  fluctuations  is  another  area  that  needs  further  study.  Even 
though  the  coefficient  on  the  Total  Budget  variable  was  negative,  it  may  not  be  cost 
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effective  for  the  Air  Force  to  increase  its  budget  in  hopes  of  making  up  the  difference 
from  decreased  overruns.  The  coefficient  on  this  variable  implies  that  by  increasing  the 
total  budget  by  1%,  the  amount  of  cost  overruns  will  decrease  by  0.17%.  The  total 
budget  of  the  Air  Force  is  roughly  $120B,  increasing  this  by  1%  would  cost  the  Air  Force 
$  1 .2B.  Overruns  are  typically  in  the  area  of  $5.3B  per  year.  If  the  total  budget  is 
increased  by  1%,  it  would  be  expected  that  overruns  would  decrease  by  0.17%  or  $9.1M. 
This  would  end  up  costing  the  Air  Force  $  1 . 19B.  Of  course,  the  reverse  is  also  true.  If 
the  Air  Force  cuts  the  budget  by  1%,  the  difference  would  be  a  savings  of  $1.19B. 

More  research  is  needed  into  this  area  to  fully  understand  the  implications  of 
budget  changes,  the  resulting  cost  overruns,  and  the  potential  savings  that  could  occur. 

A  more  micro  view  is  probably  needed.  Looking  at  the  programs  individually  and 
looking  to  see  if  the  program  had  a  decrease  in  their  budget,  what  did  that  do  to  their 
overruns?  Did  the  program  continue  spending  at  the  previous  rate  and  end  up 
overrunning  at  the  same  amount  that  they  were  cut?  In  which  case,  there  would  be  no 
savings.  Did  the  program  tighten  up  and  spend  more  prudently  and  end  up  overrunning 
less  than  they  were  cut?  This  would  result  in  an  overall  net  savings.  Or  in  the  last 
scenario,  did  the  program  end  up  overrunning  more  than  they  were  cut  because  schedules 
were  slipped  and  requirements  were  delayed.  In  this  case,  the  cut  in  funds  actually  ended 
up  costing  the  Air  Force  money. 

Also,  the  total  budget  variable  should  be  looked  at  more  closely.  This  variable 
was  significant,  however  the  other  budget  variables,  procurement  and  R&D,  were  not. 
These  are  the  variables  that  you  would  expect  to  have  an  effect  on  overruns  since  the 
acquisition  programs  are  funded  with  these  appropriations.  Is  it  because  procurement  and 
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R&D  alone  do  not  have  an  effect  on  overruns  but  the  two  together  do?  If  not,  what  is  it 
about  the  total  budget  that  is  correlated  with  overruns?  Is  it  the  MILCOM,  O&M,  or 
MILPERS  portions?  If  this  is  the  case,  causation  couldn’t  be  inferred  because  these 
appropriations  do  not  fund  acquisition  programs. 

In  conclusion,  this  research  set  out  to  determine  if  there  is  a  relationship  between 
human  capital  and  cost  overruns.  Based  on  previous  literature,  it  was  seen  that  a 
historical  link  has  been  established  between  performance  levels,  number  of  personnel  and 
education  levels.  Using  the  lessons  learned  from  these  studies  and  the  theoretical 
relationship  of  inputs  to  outputs  in  the  form  of  the  Cobb-Douglas  function,  an  analysis 
was  conducted  relating  Air  Force  cost  analysts  to  contract  cost  overruns.  From  this 
analysis,  it  was  seen  that  there  is  a  significant  relationship  between  the  number  of 
analysts,  their  education  levels  and  the  amount  of  cost  overruns  that  occur.  Depending  on 
the  cost  of  acquiring  new  personnel  and  the  cost  of  sending  them  to  school,  it  is  possible 
that  the  Air  Force  could  garner  significant  savings  by  adding  more  analysts  with  higher 
education  levels.  Other  policy  implications  from  the  study  are  the  importance  of 
maintaining  stable  budgets  and  of  accurately  predicting  inflation.  In  addition,  the 
DAWIA  act  was  seen  to  have  a  decreasing  effect  on  overruns. 
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Appendix  A:  Variance  Inflation  Factor 


Variable  VIF 


In  (Mil/Civ  Analysts) 

7.46 

DAWIA 

5.36 

In  (capital) 

4.71 

Total  Budget  Change  (2  Lags) 

4.53 

In  (Defense  Employment) 

3.61 

In  (Mil/Civ  Education) 

2.71 

Procurement  Budget  Change  (2  Lags) 

2.68 

In  (AFIT  Grads) 

2.48 

R&D  Budget  Change  (2  Lags) 

2.23 

Unexpected  Inlfation 

2.13 

Mean  VIF 

3.79 

Table  5:  Variance  Inflation  Factor 
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Appendix  B:  Variance  of  Residuals 
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Density 


Appendix  C:  Distribution  of  the  Error  Terms 


Standardized  residuals 

Figure  4:  Error  term  distribution 
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within  SOCOM.  After  leaving  Hurlburt  Field,  Capt  Feuring  was  accepted  to  the  Air 
Force  Institute  of  Technology  and  is  currently  working  on  obtaining  a  master’s  degree  in 
Cost  Analysis.  After  graduation,  Capt  Feuring  will  be  assigned  to  the  FMC  division  at 
Los  Angeles  Air  Force  Base,  California. 
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